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ABSTRACT 



This paper addresses the lack of evaluative resources for 
computer-assisted reading and writing instruction (CARWI) . It begins by 
discussing the benefits, drawbacks, and effectiveness of computer-based 
instruction. It then describes types of CARWI packages, including: (1) 

writing process software; (2) electronic handbooks; (3) interactive 
exercises; (4) comprehensive language programs; and (5) online writing labs 
(OWLs) and online reading labs. It continues by discussing cost, convenience, 
attractiveness, and effectiveness, which are the factors to consider when 
selecting CARWI programs.. It then describes two Web sites developed by the 
author to solicit and post descriptions and user testimonials of software 
packages used in writing centers nationally. The author also suggests a 
numeric rating method to evaluate the results of such questionnaires and 
concludes by discussing the advantages of CARWI. (EF) 
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TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



How Can We Assess Computer-Assisted Reading and Writing Instruction? 



Dr. Helene Krauthamer 
University of the District of Columbia 
Washington, DC 

Introduction 

Technology poses many challenges for reading and writing labs, not the least 
of which is selecting educational software to aid students in acquiring reading 
and writing skills. Packages come in a bewildering array of features, prices, 
hardware requirements, and training needs. Lab directors often do not have the 
resources available for adequately assessing the products that come to their 
attention nor is there a resource that provides evaluations of these products. 

The wide variety of computer software packages raises the question of how to 
assess computer-assisted reading and writing instruction (CARWI) and how to 
share these assessments with others. This paper will discuss the advantages 
and disadvantages of CARWI, describe some of the existing packages, and 
propose some guidelines about methods of evaluation, as well as present some 
online methods of dissemination. 

Background 

Currently, there are few, if any, resources available that evaluate CARWI, 
despite the great need for teachers and writing center staff to know what 
software to acquire and how to best direct students. There have been attempts 
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to assess computer-based instruction (CBI) in general. The Flashlight project has 
developed several evaluation instruments to help faculty and universities 
determine the effectiveness of technology in their classes and programs. 

Ehrman states that educational software should be “valuable” and “viable” where 
value is measured by “evaluation results, awards won, [and] testimonials from 
users,” and viability is described as the software being “used by enough people 
for a long enough period of time that all its investors (original developers, 
funders, publishers, institutional support staff, faculty, and students) can justly 
feel that they each have received an adequate return on their own investments in 
developing, acquiring, and/or learning to use the software” (Ehrmann “On Value, 
Viability and Success”). There are few sources to consult, however, to 
determine the value and viability of specific software packages, other than by 
casual communication with their users. 

Most agree that there are several advantages to CBI, regardless of the 
specific package. Student preference for technology is well documented, 
particularly for students with disabilities, (Fitzgerald), and merely using a 
computer allows students to learn “worldware” (Ehrmann “Asking”) - the use of 
word processing, email, and the Internet, essential skills by most standards. 
Simply sitting at a terminal provides students with the opportunity to learn 
technological skills such as manipulating the mouse, keyboarding, and following 
directions from computer screens. Students who type learn to make distinctions 
they miss when they write by hand. Learning to use educational software makes 
students more aware of computer possibilities, and, of course, more comfortable 
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around computers, a necessity for those who have limited computer experience. 
Students with various learning styles may also respond more favorably to 
multimedia lessons that newer software provides with colors, motion, and even 
sounds. 

Still, some question whether or not CBI is better than traditional instruction. 
Does CBI allow students to work longer, learn more independently, and perform 
better on tests? In 1995 Hobson characterized most programs as “nothing more 
than old workbooks and handbooks in electronic form” (215). Now, many still fit 
this description, but with the advent of CD-ROMs, more are beginning to include 
hypermedia and more interactivity. Responses, however, are still likely to be 
multiple choice or click and drag. When the response calls for actual writing, 
students need to compare their responses to model answers or ask for feedback 
from an instructor. The advantages are that feedback is usually immediate, the 
exercises are automatically scored, and, of course, responses are typed or 
clicked rather than handwritten. Essentially, computers have not revolutionized 
instruction, although students may be more motivated to receive instruction from 
a computer. 

Furthermore, computer technology allows students to work independently, to 
take more responsibility for their own learning, possibly enhancing lifelong 
learning skills. These computer programs are almost all self-paced, occasionally 
pushing students into more advanced levels, but more often letting the student 
decide which topics to work on and how intensely. Research studies (Jonassen 
as cited by Reed) point out, however, that not all students do well with more 
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learner control, notably, that low ability students do more poorly with greater 
learner control, whereas high ability students do better. Reed applies this to 
interpret some of the contradictory findings of research on the benefits of 
computer-based writing instruction and in his own research finds that students of 
low ability require more assistance such as that found in composing process 
software, whereas students of high ability do best with less assistance, using 
only word processing programs. CBI, in any case, should not mean that the 
teacher is absent. Further research on this mode of instruction in general 
(Montague and Kirk) shows that students do best and prefer the lesson when a 
teacher is guiding the instruction. (See Kysela Consultants for a review of the 
research on computer assisted instruction since 1983.) 

A pilot study (Krauthamer) did not show significant differences among the 
grades of students in an English composition class who consistently worked on 
computer writing software in the lab and those who did not. Kysela Consultants 
describe a research study at Alberta Vocational College evaluating the use of 
two reading packages, PLATO and Autoskills, on groups of adult learners. They 
report that although both instructors and students enjoyed using the software 
and benefited from the use of computers in the learning process, there were no 
significant differences between the groups using CAI and those following 
traditional instruction. They state, however, “No significant difference is 
significant. The literature indicates that for many teaching and learning situations, 
computer-assisted instruction is as effective as human instruction,” and that 
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studies also show that computer use increases both teacher and student 
motivation. 

Ehrman (“Asking”) says that the question “Do computers do a better job of 
teaching English composition than traditional methods” is a “useless question," 
that computers are here to stay and our job now is to learn to use them more 
effectively to reach more students. It is important also to develop evaluation 
instruments of the programs themselves to help us make decisions regarding 
purchases and referrals of various software packages. 

What types of CARWI are available? 

Although several classification schemes are possible, CARWI may be roughly 
divided into the following categories: Writing process software, electronic 
handbooks, interactive exercises, comprehensive language programs, and 
online writing (and reading) labs (OWLs). 

Some packages are devoted exclusively to the writing process, without 
reference to grammatical concerns. One example is Expressways, a package 
that takes the writer from the pre-writing stages through the various modes of 
rhetorical development, excluding mechanical matters. This allows students to 
start and complete their drafts directly on the program, following the cues. The 
lack of a proofreading component allows the writer to focus exclusively on the 
content and development of an assignment, permitting the writer to relax and 
write more freely. This replicates what human tutors are trained to do and may, 
in fact, do it better, insofar as human tutors often do get distracted by superficial 
details of mechanics and usage. Although we may tell students that content 
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counts most, too often in tutorials we are tempted to point out grammar errors 
that are best left for later discussions. 

Some packages are electronic handbooks, such as An Electronic Writer’s 
Reference. This package electronically replicates the contents of the grammar 
handbooks, also allowing students to do grammar and other writing activities on 
the computers, offering immediate feedback to their responses. An additional 
benefit is that this program can remain open while the student is working on a 
document, and the student can shift back and forth using the navigation bar at 
the bottom of the screen, keeping the electronic handbook readily available. 

This allows students to work directly on the topics that crop up as they are 
drafting their papers. Self-guided students can do quick grammar lessons in the 
context of their writing, thus making the instruction more meaningful than either 
traditional tutorials or isolated grammar drills. 

For students who require more intensive instruction, interactive exercises drill 
students on particular topics in reading, mechanics, and/or grammar. These 
packages range from a simple interface such as that of Writing Tutor IV, where 
the student sees only text, to very elaborate screens, such as that of Writer’s 
Solution - Language Lab and Reading Road Trip where the student is exposed 
to video displays and sound clips. Writing Tutor IV includes over 40 topics, with 
several types of exercises for each, ranging from multiple-choice to fill-in-the- 
blank. These programs can be more patient and thorough than a human tutor, 
allowing students to choose the amount of time they would like to spend on 
these lessons and providing a vast array of exercises. Also, these programs are 
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typically more interesting than the traditional workbooks and worksheets that 
used to compose most reading and writing labs. 

Some packages are comprehensive language programs, such as 
LearningPlus from the Educational Testing Service. This package includes 
modules on reading, writing, and math, preparing students to take the Praxis 
exam, as well as providing students with basic literacy needs. It also has 
practice exams and is designed for long-term instruction, possibly replacing 
workshops and classes if done on a regular basis. The advantages are that 
students can select their own times for these lessons and may work at their own 
pace, receiving accurate and personalized instruction every time. 

Finally, there are Online Writing Labs that are becoming quite prevalent and 
take many different forms, although all are available on the Internet. One of the 
most comprehensive is the Purdue OWL at http://owl.enqlish.purdue.edu , one of 
the few with a search box. Some OWLs are interactive and provide feedback to 
students who take their online quizzes, such as the University of Connecticut 
Guide to Grammar (http://ccc.commnet.edu/qrammar/ ). whereas some simply 
give information about various topics on writing. A comprehensive listing of 
OWLS is available on the National Writing Center Association website at 
http://nwca.syr.edu . There are not quite as many online reading labs, but some 
are being constructed, such as the University of the District of Columbia’s 
Reading/ Writing Connection at http://hkrauthamer.tripod.com/main.htm . The 
advantages of these are, of course, the 24/7 accessibility, the great attraction of 
the Internet, and the appeal they have for all types of learners. 
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Factors of Assessment 

When selecting CARWI, reading and writing labs may consider the following 
factors: Cost, convenience, attractiveness, and effectiveness. 

Cost is, of course, of particular concern in most labs with limited budgets. 
Some CARWI packages are free from the publishers with the adoption of their 
textbooks. For example, Expressways, Writing Tutor IV, Language Lab-Writer’s 
Solution, and Reading Road Trip were brought into our lab from publisher 
representatives. Some of these are packaged with textbooks so that students 
may have their own copies. Some are quite expensive, requiring a site license 
and user fees, such as LearningPlus. Typically, the more expensive packages 
allow better tracking and record keeping. OWLs are completely free, with 
Internet access. 

Convenience is another factor, particularly in labs where there is limited 
technological support. Some programs are easy to install and require no 
registration or training such as Writing Tutor IV, but typically these do not have 
record keeping or multimedia. Others do require technical assistance. For 
example, LearningPlus required the phone assistance of the help desk at the 
Educational Testing Service and took several months before it was running 
properly in our lab, and several other labs were unable to install it. When fully 
functioning, however, it maintains records for individual students and provides 
very comprehensive learning modules. Reading Road Trip and Writer’s Solution- 
Language Lab both require installation and CD-ROMs to run, but are installed 
easily. These programs both allow record keeping. Often, even when a program 
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is up and running, a newer version appears or sometimes a lab’s computers may 
be updated, necessitating a renewed installation. OWLs, of course, are easy to 
access on the Internet, requiring no special assistance at all. Typically OWLs do 
not have record keeping, although some publishers are designing sites that 
require the use of passwords, and these may allow students to send reports to 
their instructors. 

The attractiveness of the package to the users is also important to consider. 
Some have simple interfaces, using text only. For example, Writing Tutor IV has 
only words, no images, colors, or sounds. This is less appealing to some 
students who desire more exciting material but more appealing to advanced 
students who want to focus exclusively on skill building. Some packages use 
multimedia, with video and sound clips to accentuate the reading passages. 
Writer’s Solution-Language Lab and Reading Road Trip both utilize very dynamic 
interfaces to appeal to visual and aural learners. OWLs range from very simple 
websites to elaborate, interactive sites, and most students appreciate the 
accessibility and convenience they offer. 

Finally, labs need to consider the effectiveness of the programs they use. 
Measuring the effectiveness of any teaching method is difficult, requiring in the 
best circumstances, a carefully designed research study. Software publishers do 
not readily undertake this task (personal communication), and lab directors 
typically do not have the time to design such studies. What precisely would 
constitute an effective program? Some factors could be satisfaction levels, 
frequency of use, and increased levels of performance as measured by pre- and 
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post-test scores. Satisfaction levels are easily measured, and in most cases, 
programs receive relatively high ratings in general, particularly when compared to 
traditional lab resources such as handouts and workbooks (Krauthamer). 
Satisfaction levels for particular software packages, however, can vary according 
to level of user and also may change over time as the novelty of a particular 
package wears off. Frequency of use could be another measure, although this 
often reflects the availability of the program or the influence of teacher/staff 
referral. To adequately test levels of performance resulting from a particular 
software package, one would need to design a research study that includes a 
pre- and post-test with an experimental group and a control group, considering 
one package at a time. Kysela Consultants also describe some of the problems 
of doing controlled research studies with this population, such as difficulty in 
following up, selecting participants, and maintaining controlled settings. 

Ultimately they call for continued research, particularly as new products come 
out, to determine the best ways of integrating these programs into instruction. 

How can we evaluate CARWI? 

We still need to find ways of evaluating the software that enters the lab, to 
help lab directors decide on what they should acquire and recommend to their 
students, and also to disseminate these evaluations so that others can make 
educated decisions. 

As a start to achieve these outcomes, I have begun two web sites to solicit 
and post descriptions of the software packages used in writing centers nationally 
and include the testimonials of the users. One site is currently housed at 
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http://www.qeocities.com/hkrauthamer/maincawi.htm . Another is a Blackboard 
course site at http://www.blackboard.com/courses/CAWI . Both of these sites 
contain directions for submitting evaluations of software packages, posting a 
Standard Assessment Form that contains the following information: 

Name of Package: 

Publisher Information: 

Cost: 

Summary of Package: (Please describe what the package includes, providing a 
summary of its advantages and disadvantages.) 

How easy is it to install and use? 

How much do students use it? 

How much do students like it? 

How much do students learn? 

Other comments: 

Overall Rating: 

Date of Submission: 

Regarding grammar exercises specifically, Hacker has developed the 
following list of questions that might be asked of electronic programs: 

General questions about the program 

1 . Can the student call up an answer following each response (instead of 
having to wait until the whole exercise has been completed)? 

2. Does the program provide specific feedback explaining why an answer is 
correct or incorrect? 

3. Can the student get help - by turning to an electronic handbook or by 
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calling up charts, lists, and the like? 



Questions about an exercise set 



4. Is the skill being tested important enough for students to spend time on? 

5. Do the skills being tested match the content of the chapter or section? 

6. Are all important skills tested? For example, does the student receive 
realistic practice in both identifying errors and choosing revision 
strategies? 

7. Is on-screen reading kept to a minimum? 

8. Is the level of difficulty appropriate? Not frustratingly difficulty? Not 
insultingly simple? 

9. Is the content interesting? 



10. Is the incorrect choice a realistic error? 

1 1 . If an explanation of the answer is given, is it clear? 

12. Is the answer correct? 

This basic set of questions may serve as a checklist for reviewers of programs, 
possibly even transforming into a scale whereby each affirmative answer yields a 
point; thus, on the “Hacker Scale” a program could rate a maximum score of 12. 
Ideally, a program could have a panel of scorers, consisting of representative 
teachers and students who would provide feedback after an agreed upon 
amount of time using the particular program. Their scores could then be 
averaged to yield a rating that could be used to evaluate the nature of the 
grammatical exercises of one package with another. 



Questions about an exercise sentence 
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Conclusion 

Most agree that computer assisted learning should not replace teachers 
but should supplement instruction. In the reading and writing lab setting, CARWI 
can provide instruction where none exists, allowing students to access tutorials 
when tutors are unavailable. Perhaps the biggest advantage of CARWI is its 
potential to provide individual attention to more individuals; twenty computer 
terminals can tutor twenty students when there is only one (or no) human tutor 
available. Also, as Reed points out, with the use of multimedia and other 
innovating forms of instruction, there is finally the possibility of addressing 
individual learning styles, something not easily done in the traditional classroom. 
Of course, computers and learning software are costly, prone to failure, and, if 
not entertaining enough, run the risk of boring students as well. Computers 
cannot form meaningful relationships with students, as a tutor supposedly could. 
On the other hand, students are often enthusiastic about using computers, and 
some may even prefer working in an environment where there is no risk of 
personal embarrassment. Also, some students may trust the accuracy of 
computer tutorials over human tutorials. Frequently, the computer labs are high 
priority in schools, more so than faculty. The equipment is already available in 
many places, so the issue is mostly about selecting the most suitable software. 

If lab personnel and other software users could reflect on their use of educational 
software and pool their collective experiences, our use of technology in teaching 



would be far more effective. 



Krauthamer 



15 



Works Cited 

Ehrmann, Stephen C. “On Value, Viability and Success.” Valuable Viable 
Software in Education: Cases and Analysis . Eds. Paul Morris, Steve 
Ehrmann, Randi Goldsmith, Kevin Howat, and Vijay Kumar. The 
Annenberg/CPB Projects (1997): 26 October 1999 
<http://www.learner.org/edtech/rscheval/vvs.html>. 

“Asking the Right Questions: What Does Research Tell Us About 
Technology and Higher Learning?” Change Magazine, The Magazine Of 
Higher Learning , XXVII:2 (March/April 1995): 20-27. 26 October 1999 
<http://www.learner.org/edtech/rscheval/rightquestion.html>. 

Fitzgerald, Gail E. and Kevin A. Koury. "Empirical Advances In Technology 
Assisted Instruction For Students With Mild And Moderate Disabilities “ 
Journal of Research on Computing in Education 28.4 (Summer 1996) 526 - 
554. 21 October 1999 Masterfile Premier Item Number: 9609115675 < 
http://mdusa.lib.umd.edu:8000/WebZ/Authorize:sessionid=0 >. 

Hacker, Diana. Keynote Address at the College English Association - Middle 
Atlantic Group Annual Meeting, March 1 1 , 2000, Prince George’s Community 
College, Largo, Maryland. 

Hobson, Eric. “Taking Computer-Assisted Grammar Instruction to New 

Frontiers.” The Place of Grammar in Writing Instruction . Eds. Susan Hunter 
and Ray Wallace. Portsmouth: Heinemann, 1995. 213-224. 

Krauthamer, Helene. “A Study of the Effectiveness of Computer-Assisted Writing 
Instruction” Writing Lab Newsletter (in press). 



Krauthamer 



16 



Kysela Consultants Ltd. “Computer Assisted Reading Instruction Project: 
Technical Report." (December 1997): 15 November 1999 
< http://www.nald.ca/fulltext/avcctech/title1 .htm >. 

Montague, William E. and Frederick G. Knirk. “What Works in Adult 

lnstruction:The Management, Design and Delivery of Instruction” Navy 
Personnel Research And Development Center, San Diego, California 9 
November 1999 <http://www.nprdc.navy.mil/wworks/>. 

Reed, W. Michael. “Assessing The Impact Of Computer-Based Writing 

Instruction.” Journal of Research on Computing in Education 28.4 (Summer 
1996): 21 October 1999 Masterfile Premier 0888-6504 Number 96091 15642 
<http://mdusa.lib.umd.edu:8000/WebZ/Authorize:sessionid=0>. 




17 




U.S. Department of Education 

Office of Educational Research and 
Improvement (OERI) 

National Library of Education (NLE) 
Educational Resources Information Center 

(ERIC) 



Reproduction Release 

(Specific Document) 



CS 014 353 



L DOCUMENT IDENTIFICATION: 



Title: How Can We Assess Computer-Assisted Reading and Writing Instruction? 



Author(s): Helene Seltzer Krauthamer 



Corporate Source: 



Publication Date: May 15, 2001 



II. REPRODUCTION RELEASE: 



In order to disseminate as widely as possible timely and significant materials of interest to the educational 
community, documents announced in the monthly abstract journal of the ERIC system, Resources in 
Education (RIE), are usually made available to users in microfiche, reproduced paper copy, and electronic 
media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source 
of each document, and, if reproduction release is granted, one of the following notices is affixed to the 
document. < • 



If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the 
following three options and sign in the indicated space following. 



The sample sticker shown below will be 
affixed to all Level 1 documents 


The sample sticker shown below will be affixed to all 
Level 2A documents 


The sample sticker shown below will be affixed to all 
Level 2B documents 


PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRAN3&D BY 

Jf.. _ 


PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE, AND IN ELECTRONIC MEDIA 
FOR ERIC COLLECTION SUBSCRIBERS ONLY, 
HAS BEEN CRANED BY 

xV 


PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE ONLY HAS B££N GRANTED BY 





jr 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 


TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 


TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 


Level 1 


Level 2A 


Level 2B 


t 


t 


t 


/ 






Check here for Level 1 release, permitting * 
reproduction and dissemination in 
microfiche or other ERIC archival media 
(e g. electronic) and paper copy. 


Check here for Level 2A release, permitting 
reproduction and dissemination in microfiche and in 
electronic media for ERIC archival collection 
subscribers only 


Check here for Level 2B release, permitting 
reproduction and dissemination in microfiche only 



O 

ERIC 




Documents will be processed as indicated provided reproduction quality permits. 1 

If permission to reproduce is granted, but no box is checked, documents will be processed at Level 1 j 


/ hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to 
reproduce and disseminate this document as indicated above. Reproduction from the ERIC microfiche, or 
electronic media by persons other than ERIC employees and its system contractors requires permission 
from the copyright holder. Exception is made for non-profit reproduction by libraries and other service 
agencies to satisfy information needs of educators in response to discrete inquiries. 






Printed Name/Position/Title: Dr. Helene Krauthamer/ Director/ UDC 
Reading/Writing Lab 




Organization/Address: University of the District of Columbia, 

^ Ann A ■ • ami ■ a \ TI I T 


Telephone: 202-274-6176 


Fax: 




Connecticut Avenue, NW 
Washington, DC 20008 


E-mail Address: 
hkrauthamer@hotmail.com 


Date: May 15, 2001 





III. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC 
SOURCE): 

If permission to reproduce is not granted to ERIC, or, if you wish ERIC to cite the availability of the 
document from another source, please provide the following information regarding the availability of the 
document. (ERIC will not announce a document unless it is publicly available, and a dependable source can 
be specified. Contributors should also be aware that ERIC selection criteria are significantly more stringent 
for documents that cannot be made available through EDRS.) 




IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS 
HOLDER: 

If the right to grant this reproduction release is held by someone other than the addressee, please provide 
the appropriate name and addres s: 

Name: 

Address: 



V. WHERE TO SEND THIS FORM: 



Send this form to the following ERIC Clearinghouse: 



ERIC/REC Clearinghouse 
2805 E 10th St Suite 140 
Bloomington, IN 47408-2698 
Telephone: 812-855-5847 
Toll Free: 800-759-4723 
FAX: 812-856-5512 
e-mail: ericcs@indiana.edu 



